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Filtering a collection of samples. 

FIELD OF THE INVENTION 

The invention relates to filtering a collection of samples. Such filtering may be 
applied, for example, in image processing prior to an image-processing operation like 
up-sampling, clown-sampling or warping. 



BACKGROUND ART 

WO-A-98/06064 (attorney docket no. PHB 34.096) describes post-processing of 
a computer-generated image of a three-dimensional object for the display as a pixel 
image. A focus depth is specified either by the user or from within the system for each 

10 image frame. The image pixels are grouped into non-overlapping blocks of adjoining 

image pixels with a single respective depth value pre-specified or calculated at run-time 
for each block. Each block is extracted in turn from the image and the pixels within a 
block are filtered with a common set of filter coefficients. Padding may be applied to fill 
areas where a full filter kernel cannot be formed. An example of padding is repeating 

15 pixel edge values or "wrapping round" values from the opposite edge of the block in 
order to provide sufficient values of the right general order to construct a filter kernel 
centered on those pixels in the boundary area. 

SUMMARY OF THE INVENTION 

20 It is an object of the invention that a wide variety of collections of samples can 

be satisfactorily filtered. 

The invention takes the following aspects into consideration. Filtering a 
collection of samples usually implies that for each sample a cluster of samples is 
formed, the cluster comprising the sample itself and neighboring samples. A filtered 

25 sample is derived from the cluster, for example by applying a certain function to the 
cluster of samples. The filtered sample thus obtained is associated with the sample for 
which the cluster was formed, 

For the purpose of filtering, it is convenient to structure a collection of samples 
such that clusters of samples can easily be formed. For example, let it be assumed that 

30 a collection of samples represents a two-dimensional image. It is then convenient to 
structure the collection of samples as a matrix. 

Let it be assumed that a structured collection of samples represents a multi- 
dimensional entity having an irregular shape. This implies some that some samples are 
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valid because they belong to the entity, whereas other samples are not valid because 
they do not belong to the entity. For example, an image may be a composition of 
various arbitrary-shaped object. Let it be assumed that the image is represented by a 
collection of samples which is structured as a matrix. The collection of samples 
5 constitutes, as it were, a rectangular bounding box which contains the image. Some 
samples will belong to the image and will thus be valid, whereas other samples will not 
belong to the image and will thus not be valid. 

The following problem may occur in filtering a structured collection of samples 
which represents a multi-dimensional entity having an irregular shape. Some filtered 
10 samples will be derived from clusters comprising one or more samples which are valid 
because they do belong to the entity, and one or more samples which are not valid 
because they do not belong to the entity. That is, some filtered samples will, in effect, 
be mixtures of valid and non-valid samples. The filtered samples will thus represent the 
entity with a certain degree of distortion. This distortion will be relatively severe if non- 
15 valid samples substantially differ in value from valid samples. 

In accordance with the invention, a distinction between valid samples and non- 
valid samples is made on the basis of auxiliary data. Filtered samples which are 
associated with the valid samples, are derived exclusively on the basis of the valid 
samples. 

20 Accordingly, in the invention, it is prevented that the filtered samples which are 

associated with the valid samples, are affected by a non-valid sample. Thus, supposing 
that the collection of samples represents an entity with samples belonging to the entity 
and samples not belonging to the entity, it can be prevented that filtering the collection 
of samples entails a distortion of the entity. Consequently, the invention allows a 

25 satisfactory result in terms of distortion without this requiring that ail samples are valid 
or that any non-valid sample does not substantially differ in value from the valid 
samples. Thus, the invention allows that a wide variety of collections of samples can be 
satisfactorily filtered. 



30 implement the invention to advantage, are apparent from and elucidated with reference 
to the drawings described hereinafter. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a conceptual diagram illustrating basic features of the invention as 
35 claimed in claim 1; 



The invention and additional features, which may be optionally used to 
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Fig. 2 is a block diagram illustrating an example of an MPEG-4 image processing 
arrangement in accordance with the invention; and 

Fig 3 is a conceptual diagram illustrating how a stream of image samples can be 
formed in the MPEG-4 image processing arrangement illustrated in Fig. 2. 



DETAILED DESCRIPTION OF THE DRAWINGS 

First, some remarks will be made on the use of reference signs. Similar entities 
are denoted by an identical letter code throughout the drawings. Various similar 
entities may be shown in a single drawing. In that case, a numeral is added to the 
10 letter code so as to distinguish similar entities from each other. The numeral will be 
between parentheses if the number of similar entities is a running parameter. In the 
description and the claims, any numeral in a reference sign may be omitted if this is 
appropriate. 

Fig. 1 illustrates basic features of the invention. A collection COL of samples Si 

15 is filtered in the following manner. A distinction DIS is made between valid samples Si+ 
and non-valid samples Si- on the basis of auxiliary data AUX. Filtered samples So which 
are associated with the valid samples Si+, are derived DER exclusively on the basis of 
the valid samples Si+. 

There are various manners in which the filtered samples that are associated 

20 with the valid samples, may be exclusively derived from the valid samples. For 

example, let is be assumed that a cluster of samples is formed so as to derive a filtered 
sample. It is then possible to make a weighed combination of the valid samples in the 
cluster only, the weighing factors depending on the number of valid samples in the 
cluster. This may be realized by means of a filter of variable length, with a different set 

25 of filter coefficients for each length. However, such a filter is generally relatively costly. 
It is therefore preferable to use a filter of fixed length. 

A fixed-length filter can be used in the following embodiment. A filtered sample 
is derived from a set of filter input values which is fixed in size. Each filter input value 
is associated with a specific sample. If the sample is valid, the value of the sample is 

30 taken as the filter input value. If the sample is not valid, a padding value is taken as 
the filter input value. The padding value being derived from at least one valid sample. 
For example, the padding value could be the mean value of all valid samples. However, 
such a calculation is generally time-consuming and requires a relatively large memory. 
A relatively fast padding value calculation can be done as follows. A cluster of 

35 samples is formed. A padding value is calculated on the basis of valid samples in the 

cluster. A set of filter values is formed by taking for each valid sample the value of that 
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sample and by taking for each non-valid sample the padding value. A filtered sample is 
derived from the cluster of filter input values. 

The features illustrated in Fig. 1 may be applied, for example, to process image 
data which is composed of texture data and shape data. Such a composition will be 
5 used in a video coding standard commonly referred to as MPEG-4. In MPEG-4, the 
texture data is formed of by one or more blocks of image samples. The shape data is 
formed by a bitmap in which each bit is associated to a specific image sample. The 
value of the bit indicates whether or not the image sample concerned belongs to an 
object, or a composition of objects, which the image data represents. The bits of the 

10 bitmap, which constitutes the shape data, will hereinafter be referred to as shape bits. 

Fig. 2 illustrates an MPEG-4 image processing arrangement which recaptures the 
features illustrated in Fig. 1. The image processing arrangement comprises a receiving 
section REC, a filter arrangement FAR and an image processor PROC. In more detail, 
the filter arrangement FAR comprises a sample memory MEMSMPL, a shape memory 

15 MEMSHP, a padding-value calculator PVC, a multiplexer MUX and a filter FIL. 

The image processing arrangement operates as follows. The receiving section 
REC receives image data ID. The image data ID contains one or more blocks of image 
samples representing an object or a composition of objects. The image data ID further 
contains a bitmap associated to the one or more blocks of image samples. The 

20 receiving section REC provides a stream of image samples ISS and a stream of shape 
bits SBS. The streams are synchronized. That is, when the input section provides an 
image sample, it simultaneously provides the shape bit associated to that image 
sample. 



25 stream of shape bits SBS is synchronous with the stream of image samples ISS, both 
have the same format. Fig. 3 shows a matrix of composed of true image samples PX 
and filler image samples PF. The true image samples PX originate from the image data 
ID. The filler image samples PF have been added to the true image samples in order 
that the filter arrangement FAR provides a number of filtered image samples 

30 corresponding to the number of image samples contained in the input data ID. The 
streams of image samples ISS is formed by scanning, as it were, each row from the 
matrix concerned from the left the to the right. The scanning starts at the upper row 
and ends at the bottom row. With each filler image sample PF, the receiving section 
REC simultaneously provides a shape bit indicating that the filler image sample PF is 

35 not valid. 



Fig. 3 illustrates how the stream of image samples ISS is formed. Since the 
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The filter arrangement FAR successively forms different clusters of image 
samples and clusters of shape bits associated thereto, and derives a filtered image 
sample from each cluster of image samples. The sample memory MEMSMPL and the 
shape memory MEMSHP are first-in-first-out (FIFO) memories which receive the stream 

5 of image samples ISS and the stream of shape bits SBS, respectively. A new cluster of 
image samples is thus formed when a subsequent image sample in the stream of image 
samples ISS is written into the sample memory MEMSMPL, while deleting the image 
sample which has been first written into the sample memory MEMSMPL. Similarly, a 
new cluster of shape bits is formed when a subsequent shape bit in the stream of 

10 shape bits SBS is written into in the shape memory MEMSHP, while deleting the shape 
bit which has been first written into the sample memory MEMSHP. 

The filter arrangement FAR derives a filtered image sample from a cluster of 
image samples contained in the sample memory MEMSMPL. It does so in the following 
manner. The padding-value calculator PVC calculates a padding value PV on the basis 

15 of valid image pixels contained in the cluster. The padding-value calculator PVC can 
determine whether an image sample is valid or not by means of the shape bits 
contained in the shape memory MEMSHP. 

There are various different manners in which the padding-value calculator PVC 
may derive the padding value PV from the valid image samples. For example, the 

20 padding value may be the arithmetic mean value of all valid image samples in a cluster. 
As an other example, the padding value PV be an approximated arithmetic mean value 
being the sum of all valid samples divided by a value which is the integer to the power 
of two nearest to number of valid samples. The calculation of this approximated 
arithmetic mean value requires one or more addition and bit-shift operations only. Such 

25 operations can be carried out relatively quickly and with relatively simple hardware. As 
yet an other example, the padding value PV may be the median value of all valid 
samples in the cluster. In that case, the padding-value calculator may be implemented 
using comparators and multiplexers only. 

It may happen that a cluster of image samples stored in the sample memory 

30 MEMSMPL does not contain any image sample which is valid. In that case, the padding 
value calculator PVC may directly provide a filtered image sample. This is illustrated in 
Fig. 2 by means of a broken line. The filtered image sample may, in that case, be a do- 
not-care value or a non-valid sample which constitutes the core of the cluster, for 
example. In the following, it is assumed that there is at least one valid sample in the 

35 cluster of image samples. 
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The multiplexer MUX successively reads the shape bits contained in the shape 
memory MEMSHP. If a shape bit indicates that the image sample PX to which it is 
associated is valid, the multiplexer MUX supplies the value of image sample PX 
concerned to the filter FIL. If, however, a shape bit indicates that the image sample PX 
5 to which it is associated is not valid, the multiplexer MUX supplies the padding value PV 
to the filter FIL. 

The filter FIL makes a weighed combination of the values it receives from the 
multiplexer MUX. The weighed combination of these values constitutes the filtered 
image sample PXF. The weighing factors are filter coefficients. There is a specific filter 

10 coefficient for each position in a cluster. 

The filter arrangement FIL carries out the above-described operations for each 
different cluster of image samples and shape bits associated thereto. Accordingly, a 
collection of filtered image samples PXF will be obtained. The collection of filtered 
image samples constitutes the filtered image data FID. 

15 The image processor PROC processes the filtered image data so as to obtain 

processed image data. The image processor PROC which may carry out an image 
processing operation like, for example, up-sampiing, down-sampling or warping. 

The drawings and their description hereinbefore illustrate rather than limit the 
invention. It will be evident that there are numerous alternatives which fall within the 

20 scope of the appended claims. In this respect, the following closing remarks are made. 
There are numerous ways of physically spreading functions or functional 
elements over various units. In this respect, the drawings are very diagrammatic, each 
representing only one possible embodiment of the invention. Thus, although a drawing 
shows different functional elements as different blocks, this no by means excludes that 

25 some functional elements, or all functional elements, may be implemented as a single 
physical unit. 

Referring to the MPEG-4 image processing arrangement described hereinbefore 
with reference to Fig. 4, there are numerous manners to form different clusters of 
image samples other than supplying a stream of image samples to a FIFO memory. For 

30 example, the receiving section REC may comprise a memory in which all image samples 
and the bitmap associated thereto are stored in accordance with a certain format. The 
filter arrangement FAR may then comprise an address generator which generates 
different groups of addresses. The image samples stored under a group of addresses 
forms a cluster from which a filtered image sample is derived. 

35 Any reference sign in a claim should not be construed as limiting the claim. 
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Claims. 



1. A method of filtering a collection (COL) of samples (Si), characterized in that 
the method comprises the steps of: 

distinguishing (DIS) between valid samples (Si+) and non-valid samples (Si-) on 
the basis of auxiliary data (AUX); and 
5 - deriving (DER) filtered samples (So) which are associated with the valid samples 
(Si+), exclusively on the basis of the valid samples (Si+). 

2. A method of filtering as claimed in claim 1, characterized in that the method 
comprises the step of: 

deriving a filtered sample from a set of filter input values which is fixed in size; 
10 each filter input value being associated with a specific sample such that, if the sample 
is valid, the value of the sample is taken as the filter input value whereas, if the sample 
is not valid, a padding value is taken as the filter input value, the padding value being 
derived from at least one valid sample. 

3. A method of filtering as claimed in claim 1, characterized in that the method 
15 comprises the step of: 

forming a cluster of samples; 

calculating a padding value on the basis of valid samples in the cluster; 
forming a set of filter values by taking for each valid sample the value of that 
sample and by taking for each non-valid sample the padding value; 
20 - deriving a filtered sample from the cluster of filter input values. 

4. A filter arrangement (FAR) for filtering a collection of samples (Si), 
characterized in that the filtering arrangement comprises: 

an input circuit for distinguishing between valid samples (Si+) and non-valid 
samples (Si-) on the basis of auxiliary data (AUX); and 
25 - a filtering circuit for deriving filtered samples (So) which are associated with the 
valid samples (Si+), exclusively on the basis of the valid samples (Si+). 
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Abstract. 



A collection of samples Si, representing an image for example, is filtered in the 
following manner. A distinction is made between valid samples Si+ and non-valid 
samples Si- on the basis of auxiliary data AUX. Filtered samples So which are 
5 associated with the valid samples Si+, are derived exclusively on the basis of the valid 
samples Si + . 

Accordingly, it is prevented that the filtered samples which are associated with 
the valid samples, are mixtures of valid and non-valid samples. Such mixtures cause 
distortion, in particular if the valid and non-valid samples substantially differ in value. 
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